Analytics based on scalable hierarchical categorization of web content

ABSTRACT

Various methods and systems for performing analytics based on hierarchical categorization of content are provided. Analytics can be performed using an index building workflow and a classification workflow. In the index building workflow, documents are received and analyzed to extract features from the documents. Hierarchical category paths can be identified for the features. The documents are indexed to support searching the documents for the hierarchical category paths. In the classification workflow, a query, that includes or references content, may be received and analyzed to extract features from the content. The features are executed against a search engine that returns search result documents associated with hierarchical category paths. The hierarchical category paths from the search result documents may be used to generate a topic model of the content associated with the query. The topic model, used for web analytics, includes scores for the hierarchical category paths and for enumerated category topics.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Divisional of U.S. patent application Ser. No.14/482,514 filed Sep. 10, 2014 and entitled “Analytics Based on ScalableHierarchical Categorization of Web Content,” the entire contents ofwhich are incorporated by reference herein.

BACKGROUND

Analytics generally refers to the discovery and communication ofmeaningful patterns in data. In particular, web analytics includes themeasuring, collection, analysis and reporting of web data for purposesof understanding and optimizing web usage. Web analytics may be a toolfor business and market research. In this regard, web analytics can beused to assess and improve the effectiveness of websites. Web analyticsmay include logging web content, such as webpages, that consumers accesson particular content provider websites. However, log data provides verylittle direct information about the content or products that consumersare interested in, because of a limited understanding of the particularcontent or products on the pages visited. Further, web analytics mayimplement webpage category classifications to discover patterns in webcontent. However, category classifications can be limited when thecategory classifications lack depth and breadth in describing webpagecontent. As such, conventional web analytics fail to support theunderstanding of web content to discover consumer interest and intentinformation.

SUMMARY

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the detaileddescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used in isolation as an aid in determining the scope of the claimedsubject matter.

Embodiments of the present invention provide methods and systems forperforming analytics based on hierarchical categorization of webcontent. Analytics can be performed using an index building workflow anda classification workflow. In the index building workflow, documentsfrom a knowledge corpus are received and analyzed to extract featuresfrom the documents. The features may include leaf categories associatedwith the documents. Hierarchical category paths can be identified forthe features. The documents are indexed to support searching thedocuments for hierarchical category paths. In the classificationworkflow, a query, that includes or references content, may be receivedand analyzed to extract features from the content. The features may beexecuted against a search engine that returns search result documentsassociated with hierarchical category paths. The hierarchical categorypaths, from the search result documents, are used to generate a topicmodel of the content associated with the query. The topic model, usedfor web analytics, includes scores for the hierarchical category pathsand individual scores for enumerated category topics of the hierarchicalcategory paths.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is described in detail below with reference to theattached drawing figures, wherein:

FIGS. 1A-1D are block diagrams showing systems for performing analyticsbased on hierarchical categorization of web content, in accordance withembodiments of the present invention;

FIG. 2 is a schematic diagram showing a method for performing analyticsbased on hierarchical categorization of web content in accordance withembodiments of the present invention;

FIGS. 3A-3C are exemplary web content and topic model diagrams, inaccordance with embodiments of the present invention;

FIGS. 4A-4B are exemplary topic model diagrams for performing analyticsbased on hierarchical categorization of web content, in accordance withembodiments of the present invention;

FIG. 5 is an exemplary web page for performing analytics based onhierarchical categorization of web content, in accordance withembodiments of the present invention;

FIG. 6 is an exemplary topic model diagram for performing analyticsbased on hierarchical categorization of web content, in accordance withembodiments of the present invention;

FIG. 7 is a flow diagram showing a method for performing analytics basedon hierarchical categorization of web content, in accordance withembodiments of the present invention;

FIG. 8 is a flow diagram showing a method for performing analytics basedon hierarchical categorization of web content, in accordance withembodiments of the present invention; and

FIG. 9 is a block diagram of an exemplary computing environment suitablefor use in implementing embodiments of the present invention.

DETAILED DESCRIPTION

The subject matter of embodiments of the invention is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of this patent.Rather, the inventors have contemplated that the claimed subject mattermight also be embodied in other ways, to include different steps orcombinations of steps similar to the ones described in this document, inconjunction with other present or future technologies. Moreover,although the terms “step” and/or “block” may be used herein to connotedifferent elements of methods employed, the terms should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

For purposes of this disclosure, the word “including” has the same broadmeaning as the word “comprising.” In addition, words such as “a” and“an,” unless otherwise indicated to the contrary, include the plural aswell as the singular. Thus, for example, the requirement of “a feature”is satisfied where one or more features are present. Also, the term “or”includes the conjunctive, the disjunctive and both (a or b thus includeseither a or b, as well as a and b).

Discovering and communicating meaningful patterns in data (e.g., webdata) can be based on measuring, collecting, analyzing and reportingdata. Such operations, when performed for purposes of understanding andoptimizing web usage can be referred to as web analytics. Conventionalweb analytics methods and systems fail to support the understanding ofweb data to discover consumer interest, intent, or sentimentinformation. For example, web analytics may include logging web content,such as webpages, that consumers access on particular content publisherwebsites. However, log data provides very little information about thecontent or products of the web pages. In addition, web analytics mayinclude webpage category classifications to discover patterns in webcontent; however, category classifications can be limited when thecategory classifications lack depth and breadth in describing webpagecontent.

Embodiments of the present invention provide simple and efficientmethods and systems for performing analytics based on scalablehierarchical categorization of web content. Methods and systemsdescribed herein can be implemented with scalability to handle a growingamount of requests in a capable manner or to support the ability to beenlarged to accommodate growth. At a high level, an analytics server canprovide components to execute two different workflows. An index buildingworkflow can be performed to generate a categorizer index that indexesdocuments and corresponding hierarchical category paths of thedocuments. A classification workflow can then be performed on webcontent, based on the categorizer index, to generate topic modelscomprising hierarchical category paths.

A topic model having hierarchical topic categories provides accuracy inunderstanding web content and discovering consumer interest, intent, orsentiment. For example, a digital publisher may generate a categorizerindex using a knowledge corpus and perform a classification workflowusing their web content (e.g., collection of digital magazine articles).Topic models based on the collection of digital magazine articles may beused to better understand the collection, such that, it informs thedevelopment of new content for the collection. In addition, a contentbrowsing interface may be generated based on the topic models, suchthat, consumers accessing the web content can more easily discover andreview web content from the digital publisher.

With reference to discovering consumer interest, intent, or sentiment,hierarchical topic categories facilitate deeper and broaderunderstanding of web data in order to target users with recommendedcontent and relevant advertising. For example, categorical information(e.g., leather jackets), from a retailer web page selling a leatherjacket, provides an advertiser with insufficient information to supportperforming different types of marketing strategies. While, topic modelshaving hierarchical topic categories (e.g., retail goods->clothing &fashion->outerwear->jackets->leather jackets), generated using acategorizer index, provide rich hierarchical information to measureconsumer interest, intent, and sentiment. In particular, inferences canbe made based on consumer web log data. For example, a determinationthat a consumer is likely generally interested in outerwear can be madewhen a set of subsequently accessed web pages are associated withouterwear but not specifically leather jackets. While, an alternatedetermination can be made that a consumer is more specificallyinterested in leather jackets when a set of subsequently accessed pagesare associated with leather jackets.

Various terms are used throughout this description. Although moredetails regarding various terms are provided throughout thisdescription, general definitions of some terms are included below toprovider a clearer understanding of the ideas disclosed herein:

A feature refers to a prominent term, phrase or n-gram in a document. Afeature may be a leaf category or any other category in a selecteddocument. A feature can include identified text with high frequency in adocument.

A document context refers to a plurality of root categories for whichthe document is analyzed. The document context may be manually generatedor based on root categories of a given taxonomy. The document contextmay be associated with broad classification divisions or specificclassification divisions.

Consumer disposition refers to consumer interests, consumer intent, andconsumer sentiment. Consumer interest comprises consumer concern,involvement, or draw of consumer attention, to particular content orservice. Consumer intent comprises the purpose for performing an actionor series of actions, the end or goal that is aimed at. Consumersentiment includes consumer attitude or opinion on particular content orservice Consumer disposition (e.g., interest, intent, or sentiment) canbe directed to a variety of web content, by way of example, readershipcontent (e.g., news of specific topics) or products (e.g., clothing).

A knowledge corpus refers to a collection of structured documents. Aknowledge corpus can include a large collection of linked articleshaving category hierarchies.

A hierarchical category path refers to an arrangement of a sequence ofcategories. The hierarchical category path can start from a rootcategory to a leaf category. The root category indicates a conceptualstarting point of a classification division with successive narrowersub-categories associated with the root category.

A categorizer index refers to a construct of organized data thatsupports the highly specialized task of representing content (e.g., webcontent) in hierarchical topic categories. The categorizer index may begenerated for specific domains or a multi-purpose domain as taxonomieswith hierarchical categories. The categorizer index is more than justextracted keywords or phrases to indicate topics, but rather furthercomprises hierarchical categorization of topics. The categorizer indexcan, in particular, be implemented using category feature vectors (FV).As such, the categorizer index can also be referred to as a hierarchicaltopic categorizer index or a category feature vector index.

In operation, the categorizer index can be generated using a specificinterested party ontology (e.g., a publisher's ontology). The processmight include a manual identification of different categories, followedby, semi-supervised machine learning for hierarchy of categories. Thehierarchical categories may be a merged into the categorizer index ofthe identified hierarchical categories associated with the ontology.

In an alternate embodiment, the categorizer index can be generated usinga knowledge corpus of categories and articles. One or more corpora thatinclude linked articles and category hierarchies are contemplated withembodiments of the present invention. In one embodiment, the categorizerindex is generated using an external knowledge corpus (e.g., WIKIPEDIA).An external knowledge corpus can include over millions of individualcategories and linked articles. Categories in the external corpus may bespecifically be pruned to filter our hierarchical categories that maynot be relevant for a general purpose domain or a specific domain for aparticular interested party. The articles may be analyzed to extractfeatures that are most representative of the article. In particular, aterm frequency inverse document frequency (tf-idf) analysis can beperformed on the documents such that the extracted features from anarticle may be used to generate a collection of terms associated withthe article and also identify a subset of shortest hierarchical categorypaths of the article. The category features may be modeled in a featurevector space.

The external knowledge corpus can also be associated with a plurality oflanguages that facilitate generating the categorizer index for aparticular language. It is contemplated that embodiments of the presentinvention may leverage a multilingual corpus in generating thecategorizer index to easily localize to any number of languagessupported by the underlying knowledge corpora. By way of example, withWIKIPEDIA, the topic category ontology and training corpus is availablein 287 languages. It is further contemplated that the categorizer indexcan be periodically rebuilt, such that, it includes the most up-to-datecontent of the external corpus. As such, the categorizer index may berebuilt and deployed in near real-time and/or on demand as newcategories and subjects are added to the associated knowledge corpus.

Upon building the categorizer index, a topic model can be automaticallygenerated using web content (e.g., individual webpages or websites) orother types of input (e.g., keywords/phrases). In an exemplaryimplementation, individual pages or sets of pages identified from weblog data, for a user or user segment, can be used to generate a topicmodel of web content referenced in the web log data. The web pages maybe analyzed in near-real time or a batch of pages may be aggregated fora designated period of time and analyzed, as a user or user segmentaccesses web content. Additionally or alternatively, the web content maybe analyzed in a batch workflow that analyzes a collection of webcontent, to generate topic models. The topic models can be analyzed todiscover consumer disposition. In embodiments, an interested party canhave an on premise implementation of the analytics sever to provideanalytics functionality or subscribe to a service to provide theanalytics functionality described herein. In other embodiments, thefunctionality of the analytics server can be provided via a cloudcomputing system, where the cloud system computing provides computing asa service with shared hardware and software resources. The cloudcomputing system may also support a scalable implementation of theanalytics server.

In operation, a reference to web content (e.g., URL, or documentidentifier) and other input (e.g., keywords/phrases) may be received asqueries and analyzed using the categorizer index. Queries can begenerated as feature vector (FV) representations of the contentassociated with the queries. Content can include new and unknownwebpages using html text, page URLs, document identifiers, and otherforms of accessing and representing web content. Content can be, inparticular, selected from web log data of consumers visiting websites.Generating a query FV can include utilizing statistics-based methods toextract key text features from the content associated with the query.For example, tf-idf can be used to identify word n-grams in web contentto determine the raw frequency of text features in the web content.Other methods of extracting key text features in the web content arecontemplated with embodiments of the present invention.

The extracted text features can be used to build a FV representation ofthe query web content. The query FV can be diagrammatically representedin the feature vector space of the extracted text features from the webcontent. Each dimension corresponds to a separate text feature, suchthat, if a text feature occurs in a document, its value in the vectorspace is non-zero. The query FV can be used to identify hierarchicaltopic categories from the categorizer index. Identifying thehierarchical topic categories can be based on the highest similaritybetween the query FV and the categorizer index category FV. It iscontemplated that a specialized search engine performs the comparisonbetween the query FV and the categorizer index category FV whenidentifying the hierarchical topic categories for a topic model.

In embodiments, the topic model may further be used for tagging webcontent. In particular, the hierarchical categories may be used asdescriptive labels for web content. As such, web content can beassociated with specific hierarchical categories that describe thecontent. The hierarchical categories may be embedded as metadata of theweb page or displayed in a defined manner alongside the web content.Other variations of associating the web page with topic modelhierarchical categories are contemplated with embodiments of the presentinvention.

In operation, a subset of hierarchical categories may be selected from atopic model of a web page. Hierarchical topic categories may be scoredto indicate the relative significance of a hierarchical category to theweb content. As such, selecting the subset of hierarchical categoriesmay be based on a score associated with the topic model hierarchicalcategories. The score may be used for ranking, prioritizing, andfiltering the identified hierarchies and depth levels of hierarchy thatare used in tagging the web content. It is contemplated that selectedhierarchy categories may be weighted up or weighted down in identifyinghierarchical categories that may be used to tag web content.

In embodiments, the topic model may be formatted as a matrix in whichrows are associated with individual users or segments of user. The usersor segments of users may be associated with a selected attribute (e.g.,particular demographic information). The matrix columns can beassociated with hierarchical categories. The matrix can be used toperform analysis (e.g., regression or singular value decomposition) toidentify relationships to infer consumer disposition of new and existingusers or segments of users. For example, an advertiser may makerecommendations to a user based on analyzing hierarchical categories inthe matrix. As such, the topic model in any output format providesbroader and deeper coverage for topics in web content to supportanalytics.

Accordingly, in a first aspect of the present invention, one or morecomputer storage media storing computer-useable instructions that, whenexecuted by one or more processors, causes the one or more processors toperform a method for performing analytics based on hierarchicalcategories, are provided. The method comprises identifying a pluralityof features associated with a document in the plurality of documentsfrom a knowledge corpus. The method also includes identifying a documentcontext analyzing the plurality of features. The method further includesgenerating a hierarchical category path for a selected feature from theplurality of features and the document context. The method also includescreating an index entry for the document associated with the selectedfeature and the hierarchical category path.

In a second aspect of the present invention, a computer-implementedmethod suitable for use in a computing environment utilizing a processorand a memory for performing analytics based on hierarchical categoriesis provided. The method includes receiving a classification engine querythat references web content. The method also includes identifying aplurality of features associated with the web content referenced by theclassification engine query. The plurality of features is represented asfeature vectors in a vector space of the plurality of features. Themethod also includes transmitting the feature vectors as search enginequery feature vectors. The method includes receiving documentsidentified based on the search engine query feature vectors. Thedocuments are identified based on a similarity between the search enginequery feature vectors and feature vectors associated with documentsindexed in the categorizer index. The method also includes identifyinghierarchical category paths associated with the documents. The methodincludes creating a topic model comprising the hierarchical categorypaths. The method also includes generating topic model scores associatedwith the hierarchical category paths. The method further includescommunicating the topic model comprising the hierarchical category pathsand the path scores.

In a third aspect of the present invention, a system for performinganalytics based on hierarchical categories is provided. The systemincludes a categorizer index build component configured for identifyinga plurality of features associated with a document in the plurality ofdocuments; identifying a document context for analyzing the plurality offeatures; generating a hierarchical category path for a selected featureusing the plurality of features and the document context; and creatingan index entry for the document associated with selected feature and thehierarchical category path. The system further includes a classificationengine component configured for: receiving a classification enginequery; identifying a plurality of features associated with theclassification engine query; transmitting a feature from the pluralityof features as a search engine query; receiving a document identifiedbased on the feature as the search engine query; identifying ahierarchical category path associated with the document.

Turning now to FIG. 1A, an example analytics infrastructure(“infrastructure”) 100, in which embodiments of the present inventionmay be performed, is shown. The infrastructure 100 is an example of onesuitable infrastructure. The infrastructure 100 should not beinterpreted as having any dependency or requirement related to anysingle module/component or combination of modules/components illustratedtherein. Each may comprise a single device or multiple devicescooperating in a distributed environment. For instance, components maycomprise multiple devices arranged in a distributed environment thatcollectively provide the functionality described herein. Additionally,other components not shown may also be included within the networkenvironment.

The infrastructure 100 may include client devices 102, web contentserver 104, knowledge corpus server 106, analytics server 108, andnetwork 110. The network 110 may include, without limitation, one ormore local area networks (LANs) and/or wide area networks (WANs). Suchnetworking environments are commonplace in offices, enterprise-widecomputer networks, intranets and the Internet. Accordingly, the network110 is not further described herein.

Any number of client devices and servers may be employed in theinfrastructure 100 within the scope of embodiments of the presentinvention. Each may comprise a single device/interface or multipledevices/interfaces cooperating in a distributed environment. Forinstance, the analytics server 108 may comprise multiple devices and/ormodules arranged in a distributed environment that collectively providethe functionality of the performing analytics based on scalablehierarchical categorization of web content. Additionally, othercomponents/modules not shown also may be included within theinfrastructure 100.

It should be understood that this and other arrangements describedherein are set forth only as examples. Other arrangements and elements(e.g., machines, interfaces, functions, orders, and groupings offunctions) can be used in addition to or instead of those shown, andsome elements may be omitted all together. In particular, otherarrangements may support scalable implementations of embodiments of thepresent invention. The infrastructure 100 can be scalable to handle agrowing amount of requests in a capable manner or to support the abilityto be enlarged to accommodate growth, as described in more detailherein.

Further, many of the elements described herein are functional entitiesthat may be implemented as discrete or distributed components or inconjunction with other components, and in any suitable combination andlocation. Various functions described herein as being performed by oneor more entities may be carried out by hardware, firmware, and/orsoftware. For instance, various functions may be carried out by aprocessor executing instructions stored in memory.

The client device 102 may include any type of computing device 900described below with reference to FIG. 9, for example. The client device102 can provide access to different web content, for instance, web sitesthat comprise a set of related web pages typically served from a singleweb domain. A website is hosted on at least one web server (e.g., webserver 104), accessible via a network (e.g., network 110) such as theInternet or a private local area network through an Internet addressknown as a Uniform resource locator (URL). Client device 102 can in thealternative provide access to the analytics sever 108 and output of theanalytics server 108 from performing analytics on web content, asdescribed further herein.

The web server 104 is configured to store, process and deliver web pagesto client devices 102. The communication between client devices 102 andthe web server 104 takes place using the Hypertext Transfer Protocol(HTTP). Pages delivered are most frequently HTML documents, which mayinclude images, style sheets and scripts in addition to text content. Auser agent, commonly a web browser or web crawler, initiatescommunication by making a request for a specific resource using HTTP andthe server responds with the content of that resource. Web content caninclude posted content and selling of contextual advertising eitherthrough direct sales or through an advertising network,e-commerce—products or services purchased directly through a website,advertised products or services available at a brick and mortarbusiness, and freemium—basic content is available for free but premiumcontent is paid. Other types of web content are contemplated withembodiments of the present invention.

Access to web content on web server 104 can be logged. Logging access toweb content can be performed using several different types of techniquesthat generate web access logs. In particular, a web server can maintaina history of page requests. Web logs can be in standard World WideConsortium (W3C) format—the common log format or other proprietaryformats. Information about the request, including client IP address,request date/time, page requested, HTTP code, bytes served, user agent,and referrer are typically added. Web logs based on consumer access fromclient devices can be used in analytics in accordance with embodimentsof the present invention.

A knowledge corpus server 106 can facilitate generating the categorizerindex 130 of FIG. 1B. The categorizer index 130 can be generated usingthe knowledge corpus server 106 that stores hierarchical categories andarticles. In one embodiment, the categorizer index 130 is generatedusing an external knowledge corpus (e.g., WIKIPEDIA). An externalknowledge corpus can include over millions of individual categories andlinked articles. The external knowledge can also be associated with aplurality of languages that facilitates generating the categorizer indexfor a particular language. It is contemplated that embodiments of thepresent invention may leverage multilingual corpora in generating thecategorizer index 130 to easily localize to any number of languagessupported by the underlying knowledge corpora. By way of example, withWIKIPEDIA, the topic category ontology and training corpus is availablein 287 languages. One or more corpora that include linked articles andcategory hierarchies are contemplated with embodiments of the presentinvention.

The analytics server 108, further described in FIG. 1B, facilitates thediscovery and communication of meaningful patterns in data. Inparticular, the analytics server 108 may operate with additionalcomponents and modules in the infrastructure 100 to perform analyticsbased on scalable hierarchical categorization of web content. Theanalytics server 108 may include a categorizer index build component 120and a classification engine component 140 that facilitate implementingprocess workflows for generating the categorizer index 130 and topicmodels, respectively. The analytics server may further include a taggingcomponent 200 for associating web content with hierarchical categoriesas descriptive labels, and analytics application for performingadditional processing based on analytics server output. In this regard,the analytics server 108 generates different types of output includingthe categorizer index 130 based on the knowledge corpus 106, a topicmodel of hierarchical categories based on both the categorizer index 130and a query FV, and tagged web content based on the hierarchicalcategories of the topic models.

The generated categorizer index 130 is a construct of organized datathat supports the task of representing content (e.g., web content) inhierarchical topic categories. The categorizer index 130 can beimplemented as category feature vectors (FV) in an index. Topic modelsare generated from the categorizer index 130 and are analyzed toidentify consumer disposition such that interested parties can developmarket strategies including segmenting and targeting. In one example,the invention can also be used by interested parties (e.g., digitalpublishers) in building models of consumer disposition. The hierarchicaltopic categories of topic models generated from a digital publisher'sweb content can be used to model content of articles that readers areinterested in. Similarly, target analysts that need to understandproduct interests and product intentions of web content, can use thehierarchical topic categories of topic models to discover thisinformation.

With continued reference to FIG. 1B, the categorizer index buildcomponent 130 is responsible for generating the categorizer index 130.The categorizer index 130 can include highly diverse and detailedcategory topics. The categorizer index build component 130 may implementan index building workflow in building the categorizer index 130. Thecategorizer index build component 130 may implement a serviceapplication that performs the functions of the pre-processing workflow.By way of example, the service application provides a processinginterface to send and receive service requests to internal and externalcomponents and modules in building the categorizer index.

The index building workflow utilizes knowledge corpus server 106,knowledge corpus preparation module 122, category features extractionmodule 124, index creation module 126, and index search engine module128. The knowledge corpus server 106 comprises linked documentsassociated with categories. Documents can be associated with one or morecategories. In one embodiment, the WIKIPEDIA corpus, that includesdocuments and human-created categories associating the documents tocategories, can be used during the index building workflow. It iscontemplated that a plurality of corpora may be used in embodiments ofthe present invention to generate the categorizer index 130 to includeinformation from each of the corpora. The index building workflow canidentify a specific language for which the categorizer index is beingbuilt when the knowledge corpus is associated with several languages.

In operation, the knowledge corpus preparation module 132 accesses theknowledge corpus data from the knowledge corpus server 106. Theknowledge corpus preparation module 132 is responsible for preparing thecorpus data for next steps in the workflow. For example, the knowledgecorpus preparation module 122 can parse and clean up markup tags toextract information from the knowledge corpus. At this step, it may bepossible to identify the total number of articles and categories. Thisinformation on total number of articles and categories may be used fordetermining and evaluating updates to the knowledge corpus. As such, thecategory index may be rebuilt upon determining updates to the knowledgecorpus.

The features extraction module 134 is responsible for processing thedata from the knowledge corpus preparation module 122. Processing thedata can include identifying feature vectors for each of the articles(or document) in the corpus data and corresponding category hierarchies.In the corpus data, an article include can a several different features,such as, leaf level categories. A relevant hierarchical category pathcan be generated from root categories to leaf categories for eachfeature vector associated with an article in the corpus data.

In operation, root level categories can be identified. The root levelcategories define the document context. A root category can indicate aconceptual starting point of a classification division. A root categorycan include the broadest representation of a classification divisionwith successive narrower subcategories associated with the rootcategory. Identifying root level categories may be for a specific domainor a general purpose domain. Identifying the root level categories mayalso be a manually curated process. Exemplary root level categories caninclude: Agriculture, Arts, Business, Chronology, Concepts, Culture,Education, Environment, Geography, Health, History, Humanities, Humans,Language, Law, Life, Mathematics, Medicine, Nature, People, Politics,Science, Society, Sports, Technology. Other types of selected root levelcategories are contemplated with embodiments of the present invention.

Next, using the leaf category of a feature vector for a term in anarticle in the corpus data, k shortest paths from each leaf category toreach any of the root categories are identified. By way of example, aWIKIPEDIA article (e.g. about Barrack Obama), may be parsed and toextract the immediate WIKIPEDIA categories (leaf categories) associatedwith the article (e.g. “President of United States”, “Harvard Law SchoolAlumni”) as features. A hierarchical category path of this article fromroot categories (e.g., Politics, Education) to leaf categories may beidentified using the WIKIPEDIA corpus. In this regard, if an article hasN categories, the k×N category hierarchies from the root categories forthat article can be generated.

The index creation module 126 is responsible for creating an optimizedfeature vector for a selected feature. The index creation module 126receives the output from the category features extraction module 136 andcreates the optimized feature vector for the selected feature. Inoperation, each filtered tf-idf feature list of a selected document isconfigured into a search-platform-formatted document associated with asearch platform (e.g., APACHE SOLR). The search-platform-formatteddocument includes fields for title, text, and feature categories. It iscontemplated that the title field can be indexed without removing stopwords. Stop-words refer to selected words which are filtered out priorto, or after processing of natural language data. Stop-words cancontribute a significant role in the title (e.g., stop words in movietitles—THE DARK KNIGHT) whereas stop words are removed from article textbefore term feature vector creation and indexing is performed. The indexcreation module 136 is further responsible for dividing the featurevector index into N parts. The division is contemplated in embodimentsto support a multi core search feature of the search platform, asdiscussed in more detail below. The index creation module 126 generatesindex entries, in the categorizer index 130, for the documentsassociated with the feature categories and the hierarchical categorypaths.

With continued reference to FIG. 1B, the index search engine module 128is responsible providing a search platform (e.g., APACHE SOLR). Theindex search engine module 126 retrieves data from the categorizer index130 comprising a document category term feature vector index to supportsearch functionality. It is contemplated that embodiments of the presentinvention may be implemented in a scalable manner. The infrastructure100 can support the categorizer index 130 using multiple cores acrossprocessing nodes, or a single core on one node. In a multi-core set up,one main search engine instance and one or more other secondary searchengine instances can be started (e.g., APACHE SOLR core). The maininstance can act to run an orchestration/coordination process.

In embodiments of the present invention, an orchestration and querydispatch system can implemented to distribute in parallel any query toall nodes. The orchestration and query dispatch system can merge theresults returned from all the search engines modules. Since a singlenode searches only a small portion of the categorizer index 130 and eachnode searches in parallel with other nodes, the overall search time isreduced in proportion to a factor of M in the multi-core configurationwhere there are M nodes. This speed up is due to smaller categorizerindex partitions and higher bandwidth in the request pipeline.

With continued reference to FIG. 1B, the classification engine component140 is configured to generate a topic model for a received query 142. Inparticular, the classification engine component 140 can receive query142 associated with web content and generate topic model categories forthe web content. Query 142 can be associated with short text queries(e.g., keyword, phrases) or full document web pages when sent to theclassification engine component 140.

The classification engine component 140 may implement a classificationworkflow in building the topic model. The classification enginecomponent 140 may implement a service application that performs thefunctions of the classification query workflow. By way of example, theservice application provides a processing interface to send and receiveservice requests to internal and external components and modules inclassifying the query 142. The classification engine component caninclude a query features extraction module 144, a query creation module144, a results analyzer module 148, and a scoring module 150.

In operation, query 142 can be submitted to the classification enginecomponent 140. The query features extraction module 144 is responsiblefor extracting features from the text of the query input 142. By way ofexample, a text engine n-gram extraction service (e.g., SEDONA) can beused to extract text n-grams. In one embodiment, up to five grams areselected and the text engine extraction server can be configured tobreak the n-gram upon encountering stop words. The query featuresextraction module 144 can be configured to transmit the features foradditional processing.

The query creation module 146 is responsible for identify features foradditional processing. In one embodiment, a subset of the features canbe determined. For example, the query creation module 146 may identifytop k (e.g., 40) features. The query creation module 146 may sort thefeatures according to term frequencies. The query creation module 146can also construct queries to be transmitted for searching (e.g., on asearch platform—APACHRE SOLR) by weighing each term with a square rootof its term frequency.

It is contemplated that short and long documents can be handleddifferently. In particular, short queries consist of few very conciseterms and match indexed titles better than an entire article text asrepresented by the indexed article feature vector. Long queries, on theother hand, consist of more topics and themes and thus better match thefeature vector from the article body text indexed by the categorizerindex. During the index build workflow, an article title metadata isused for the short document text and article body text is used for thelong document text.

In operation, for a long document, a query to be executed using thecategorizer index via the index search engine module 146 is constructedfor the text field. For short documents, a hybrid approach can beemployed in which a query is executed separately for both the title andtext fields and the top documents are identified for both searches andresults analyzed in the result analyzer module 150. As such, a query (orqueries for short document) is submitted to the index search enginemodule 126, which then generates the top matched documents.Distinguishing a short document from a long document can be based on anempirically determined term count threshold.

The result analyzer module 148 is responsible for analyzing documents toidentify corresponding category hierarchy paths. In particular, theresult analyzer module 148 analyzes the top matched documents from theindex search engine 128. In embodiments, the results of the long andshort documents can also be handled differently. It is contemplated thatfor long documents, results correspond to the search executed on textfield. Hierarchical category paths can be identified for top matcheddocuments. For short documents, the results correspond to the searchesexecuted in the text field and the title filed. As such, top documentsare identified for both of the searches by using various approaches,such as, jaccard similarity coefficient or simple aggregation. Once thetop documents are identified, their corresponding category hierarchypaths can be found and scores aggregated similar to long documents. Topcategories identified from the result analyzer module 148 can beprovided as output categories for the input document. The scores for thecategory hierarchy paths can also be identified for top matcheddocuments using the scoring module 150.

The scoring module 150 is responsible for generating scores associatedwith hierarchical categories. The scores may be used as indicators ofconsumer disposition. In operation, for a selected webpage or selectedby a URL, or a basic query given by keyword terms, the computed topicmodel TM [URL/query] is an ordered set of M tuples consisting of:(hierarchical category topic path P, path score ωP).

TM [URL/query] = {           (hierarchical category topic path 1, pathscore ω_(P1)),           (hierarchical category topic path 2, path scoreω_(P2)),             :           (hierarchical category topic path M,path score M)          }

A hierarchical category topic path itself is a full enumeration ofcategory topics from the classification ontology from root category toleaf:

Hierarchial  Category  Topic  Path = [Root-category, Sub-category, Sub-Sub-Category, ¨] = [C₁, C₂, …  , C_(i), …  , C_(q)]q  designates  the  leaf  level.

To compute the topic model scores for a single page, individual topiccategory scores can be calculated by summing the score contributionsfrom the lowest (leaf) levels in the category path for each immediateparent, then propagating this information up to the root level of theontology. At each stage of the score summing process, individual totalcategory occurrence counts are normalized by the total number ofcontributing subcategories to given a contribution score between (0.0and 1.0) with the sum of score contributions always totaling 1.0 foreach parent category. If ω_(Ci) designates the score for individualcategory C_(i) then ω_(Ci) is computed by summing the ω scores for eachj'th child (sub category) of C_(i) indicated in the equation below byC_(ij):

$\omega_{Ci} = {{k^{- 1} \times \alpha_{\ell}{\sum\limits_{j = 1}^{k}\omega_{Cijt}}} = \left( {\omega_{{Ci}\; 1} + \omega_{{Ci}\; 2} + \ldots + \omega_{Cij} + \ldots + \omega_{Cik}} \right)}$

The above equation gives the scores for topic categories for a singleweb page or query. To compute the aggregate topic model across a giventime window with T_(N) models for each time sample t, model scores foreach category C_(it) are aggregated

$\begin{matrix}{\omega_{CiT} = {{\left( {{TN} \times k} \right)^{- 1} \times \alpha_{\ell}{\sum\limits_{t = 1}^{T_{N}}{\sum\limits_{j = 1}^{k}\omega_{Cijt}}}} = \left( {\omega_{{Ci}\; 11} + \omega_{{Ci}\; 21} + \ldots +} \right.}} \\{\left. {\omega_{{Cij}\; 1} + \ldots + \omega_{{Cik}\; 1}} \right) +} \\{\left( {\omega_{{Ci}\; 12} + \underset{::}{\omega_{{Ci}\; 22}} + \ldots +} \right.} \\{\left. {\omega_{{Cij}\; 2} + \ldots + \omega_{{Cik}\; 2}} \right) +} \\{\left( {\omega_{{Ci}\; 12} + \omega_{{Ci}\; 22} + \ldots +} \right.} \\\left. {\omega_{{Cij}\; 2} + \ldots + \omega_{{CikT}_{N}}} \right)\end{matrix}$Exemplary scores computed by these two equations are shown in FIGS. 4Aand 4B.

It is contemplated that the α_(l) coefficient may be used if anon-uniform score weighting scheme is desired to overweight orunderweight category scores at any level l, otherwise this coefficientis set to 1.0. When !=1.0, the value of α_(l) used corresponds to thehierarchical path level at which Ci occurs. The scoring algorithmdescribed above is meant to exemplary and not limiting. As such, othervariations and combinations of scoring algorithms are contemplated withembodiments of the present invention.

The tagging component 200 can be used to tag web content with topicmodel hierarchical categories as descriptive labels. The web content canbe associated with specific hierarchical categories that describe thecontent. The hierarchical categories may be embedded as metadata of theweb page or displayed in a defined manner alongside the web content. Webcontent may be received at the analytics server 108 and analyzed usingthe categorizer index 130 to generate a topic model. The taggingcomponent 200 receives the web content and the corresponding topic modelin order to tag the web content with the hierarchical categories.

In operation, a subset of hierarchical categories may be selected from atopic model of a web page. Hierarchical categories may be scored usingthe scoring module 150 to indicate the relative significance of ahierarchical category to the web content. As such, selecting the subsetof hierarchical categories may be based on a score associated with thetopic model hierarchical categories. The score may be used for ranking,prioritizing, and filtering the identified hierarchies and depth andbreadth levels of hierarchy that are used in tagging the web content.

The tagging component 200 may implement a plurality of tagging rules fortagging web content with particular hierarchical categories. An entirewebsite, individual pages, and or portion of web page content may betagged with hierarchical categories. The tagging rules can be associatedwith specific domains or particular characteristics of the web content.For example, tagging rules for retailer content may be different fromdigital magazine publishers. Tagging rules may specifically be based onthe scores associated with topic model categories. For example, athreshold score may be defined for the depth or breadth of a topicmodel, such that, a hierarchical category that is used to tag contentincludes at least a minimum score. The threshold score may be associatedwith a particular depth level or breadth level. In the alternative, aparticular number of levels of depth and breadth in categories may beselected for tagging web content. It is contemplated that selectedhierarchical categories may be weighted up or weighted down inidentifying hierarchical categories that may be used to tag web content.Other variations and combinations of the tagging rules are contemplatedwith embodiments of the present invention.

The analytics server 108 can communicate the generated topic modelassociated with query 142 to an analytics application 210. Analyticsapplication 210 can be associated with interested parties that run theanalytics applications to understand consumer disposition. Analyticsapplication 210 can aggregate hierarchical category paths and scoresinto a multi-level interest model that reflects the user or usersegments topic interests for a given time window for which web pagelevel topic models are computed. It is contemplated that the analyticsapplication 210 may receive matrix representation of the topic model toperform analysis. For example, consumer disposition may be discoveredbased on analyzing the matrix representations of the topic model.Analytics application 210 may also analyze near-real time or batchprocessed user web pages to make inferences about consumer disposition.For example, inferences may be made based on analyzing webpagehierarchical category paths and corresponding scores, which isdistinguishable from merely analyzing a collection of terms orcategories without hierarchical relationship, associated with a webpage.

With reference to FIGS. 1C and 1D, in embodiments, the functionalitysupported using the analytics server is scalable to support high webcontent query request volumes. In operation, a scalable implementationleverages multiple processing nodes for parallel implementations of thecategorizer index component 120 and classification engine component 140that utilize partitioning and distribution of the categorizer indexacross multiple processing nodes, as discussed more detail below. Ascalable implementation supports parallel text classifier constructionthat allows new topical information to be quickly incorporated into thesystem. New information can be based on updates to the knowledge corpusor directly learned from content web pages. Parallel run-time queryexecution to compute classifications allow a high volume of webpageanalysis requests to be sent to the system and topic model results to bequickly computed for each request.

A scalable implementation can include a parallel categorizer index buildcomponent 160 and parallel classification engine components 180. Theparallel categorizer index build component 160 utilizes a normalizedknowledge corpus 162, category FV extraction and partitioning module164, create FV index partition module 168, query FV search & similaritymodule 170, and orchestration of web page or query classificationrequest module 172. It is contemplated that in embodiments that theparallel categorizer index build component 160 can perform the functionsof the categorizer index build component 120. The parallel categorizerindex build component 160 is additionally responsible for updating thecategorizer index periodically based on a predefined period of time. Thepredefined period of time can be defined as once every few days.

The parallel classification engine component 180 utilizes a category FVpartitioning module 182, a query FV search & similarity module 184, aweb page text classification request orchestration module 186, a targetworker node 188, and worker node task manager 190. It is contemplatedthat in embodiments that the parallel classification engine component180 can perform the functions of the classification engine component140. The parallel classification engine component 180 can be responsiblefor categorizing and tagging new documents and queries submitted to thesystem. The parallel classification engine component 180 may executeclassification at run-time.

Embodiments of the present invention may be implemented via a cloudcomputing platform. In particular, a scalable implementation can beimplemented using a cloud computing platform that comprises componentsdescribed herein in a distributed manner. A cloud computing platform mayspan wide geographic locations, including countries and continents. Thecloud computing platform may include nodes (e.g., computing devices,processing units, or blades in a server rack) that are allocated to runone or more portions of the components of the present invention.Components may be partitioned into virtual machines or physical machinesthat concurrently support functional portions such that each functionalportion is able to run on a separate virtual machine.

In operation, the parallel categorizer index build component 160 canimplement a parallel categorizer index build workflow. During theworkflow, the knowledge corpus that contains category ontology andassociated exemplar documents is submitted to the category featureextraction and partitioning module to produce category index partitionsacross N nodes each assigned to a processing core. The parallelclassification engine can implement a parallel classification workflowthat retrieves the category feature vector index formed by N partitions.The category feature vector index 130 is made available to one or morequery index engines which submit queries against the build index.Requests to each index engine are dispatched by an orchestration system.The computation structure of the scalable implementation provides forhorizontal scaling that can support massive category indexes and a largenumber of concurrent application requests.

In a scalable computing platform embodiment, the scalable computingplatform can include a worker node architecture that providesscalability through a task manager which dispatches webpageclassification requests in a production workflow to each worker nodeclient. A worker node 188 client then communicates a request URL to theanalysis engine which fetches the web page content using the input URL,performs html extraction, sentence segmentation, term tokenization, andn-gram extraction to create input query or document FVs. The resultingquery/document FV is dispatched to the classification orchestrationmodule 186 that uses an available search and similarly module tocomputer the hierarchical topic categories.

Turning to FIG. 2, embodiments of the present invention can further bedescribed by way of example to provide details on analytics based onhierarchical categorization of web content. Hierarchical categorizationcan be used in two broad exemplary implementations—digital publishingand market strategy—as discussed in examples below. With reference todigital publishing, in one example, a content publisher (e.g., NATIONALGEOGRAPHIC) may be interested in learning a distribution of topics inthe content provided by content publisher website. Identifying the topiccategories can help the content publisher make decisions on new anddifferent content that could be added to the website.

In another example, a consumer (e.g., reader of web content on NATIONALGEOGRAPHIC) may be interested in learning the topics available on agiven website. Knowledge of these topics allows the user to easilybrowse through content available on the website and discover contentthat would otherwise have been difficult to find or not found at all. Inparticular, a hierarchical categories representation of the topic on awebsite may be presented via the website itself to the consumer. Thehierarchical categories in particular may be selectable to theindividual web page or web pages associated with the category. As such,the consumer may browse the website directly from the hierarchicalcategory visualization of the website.

In yet another example, a content publisher may be interested in findingout what topics readers are most interested in over time. Knowledge ofthese topics allows content publishers to recommend additional relatedcontent and advertising. With each of the above example, embodiments ofthe present invention provide a categorizer index which may be used toanalyzed web content to generate topic models of hierarchical topiccategories. It is contemplated that analysis of the web content can beperformed at different times. For example, when new pages are published(i.e., pages that have not been previously crawled and analyzed) or whenchanges to existing pages are detected. Other mechanisms for triggeringanalysis of web content are contemplated with embodiments of the presentinvention.

In operation, an analytics server 108 identifies and extracts topicsfrom web content 202 topics 204, places 206, people 208 and brands 210.Content text can be retrieved from several different sources, including,one or more website crawler technologies as described in AdobePatent/U.S. patent application Ser. No. 14/054,291, incorporated hereinin its entirety. A web crawler (e.g., a web crawler module) may be aninternet bot that systematically browses selected web content. Contenttext can also be retrieved from Portable Document Format (PDF) documentshosted on a publisher's website.

With reference to FIG. 3A, webpage 300 can be encoded using html andcontains content text that is extracted and then sent to query featuresextraction module 144. Extracted features in the form of word n-gramsare then used to query the categorizer index 130. The categorizer index130 consists of example documents associated with hierarchicalcategories. Content text, of the sample NATIONAL GEOGRAPHIC web content302, is indicated by html markup, such as, <p> tags and is firstextracted by a website crawler (not show). It is contemplated thatboilerplate text, graphics, and non-relevant text can be ignored orfiltered out by the text extraction module. The output of the queryfeature extraction module 144 is used to generate topic models via theclassification engine component 140. The classification engine component140 produces hierarchical categories that form a topic model for thewebpage as shown in FIG. 3B.

In FIG. 3B, the category nodes are shown from left to right and from topto bottom. Numeric values from the graph branches indicate relativecontribution of the category to the topic model. By way of example,0.254 for “People and Society” 302 indicates that roughly 25% of thetopic model for the webpage relates to this general category. The “47”304 at the end of the path may indicate the raw term frequency (TF) orthe count of the number of times that a feature/term was encountered inthe query document. In alternative implementations, this TF score may bescaled or normalized to compensate for large variations in documentlength.

It is contemplated that topic models can be generated using differentsizes of categorizer indexes having different sets of example documentsfor selected domains. Multiple categorizer indexes or ontologies can beused for the same query FV. Categorizer index 130 may be specificallyconfigured for particular interested parties such that the categorizerindex is tailored to a particular ontology. Web content may be analyzedbased on a categorizer index not specifically built for the web content.Other variations and combinations of categorizers and running queries onone or more categorizers are contemplated with embodiments of thepresent invention.

A topic model can further comprise a set of categories organized into ahierarchical category structure where category nodes in the tree towardsthe root indicate more general categories, and nodes closer to the leaflevels indicate increasing more specific topics. The depiction of thetopic model, based on categories in the categorizer index, in the mannerset forth in FIG. 3B is merely exemplary and other graphical depictionsof topic models based on hierarchical category data in a categorizerindex are contemplated with embodiments of the present invention.

With reference to FIG. 3C, FIG. 3C illustrates a more complex topicmodel based on using a general purpose categorizer index. The generalpurpose categorize can be generated using knowledge corpus of documentsfrom an external corpus spanning over a million different categories. Anexternal corpus of this size provides an ontology structure that ispre-existing, automatically maintained and kept up-to-date, andavailable in multiple languages. In this regard, the knowledge corpuscan be processed and learned from in a more robust and efficient manner.It is contemplated that the same query may be executed again a specificdomain categorizer index and a general purpose domain categorizer indexfor comparative analysis topic models to discover consumer disposition.

With reference to FIGS. 4A and 4B, web content can be specificallyconverted into query FVs and executed against the categorizer index 130to generate topic models for the web content. As such, topic models canbe generated for query FVs generated from entire webpages of a websiteor a set of pages that form the set website. A topic model can begenerated for a working set of pages that are of interest to aninterested party. By way of example, a topic model can be constructedfor over 600 pages for a snapshot of the NATIONAL GEOGRPAHIC website, aspartially depicted in FIG. 4A. Category nodes are indicated by circleswith labels. The labels on the left are used to represent more genericparent topic categories with nodes toward the right become increasingmore specific. The output provides a way for consumers to view andnavigate the topical contents of a website and also provide interestedparties (e.g., content publisher) with a topic inventory whenapproaching new advertisers to demonstrate relevance of the advertiser'sproduct to the website.

In another embodiment of the present invention, the pages visited by aspecific user or user segment can be analyzed and modeled in a topicmodel having hierarchical classification structures, as partiallydepicted 4B. For example, the 20+ most recent pages of a selected usercan be analyzed and the topic model, shown in FIG. 4B, is thenaggregated to indicate the key topics of interest to the particularuser. In another example, the topic model may be for pages seen by auser for a particular date range. Other variations and combinations ofidentifying a set of webpages associated with a visitor during a definedperiod of time are contemplated with embodiments of the presentinvention. The topic model can provide a comprehensive at-a-glancesummary of a user or a user segment disposition for certain topics whichan interested party can easily understand.

To compute the topic model for user or user segment disposition, theinvention determines a core topic model of all the topics on the webpages that a visitor views in a particular window of time. In theexample in FIG. 4B, for a one week period, at the top level of the topicmodel, the “People & Society” 400 topic category received the highestscore of 0.1475, followed by “Science” 402 with a score of 0.1366, etc.At the next subcategory level under “People & Society” 400, mentions of“society” 404 and topic “Social Sciences” 406 received the highest scopewith contributions of 0.2627 and 0.2581, respectively. As such, thescores in parenthesis in FIGS. 4A and 4B indicate the relativecontribution of a category to the parent category. Scores for allimmediate categories under a parent category (terms that form a column)can be configured to sum to 1.0 at the parent level (on the immediateleft). In the particular case of FIG. 4B, there was only one childcategory matched (Anthropology), therefore the score contribution to theparent level is 1.0 or 100%.

Using the topic model for each user or user segment, topical interestfor the user or user segment can be traced down to detailed leaf levelsof the topic model tree for direct content and ad recommendations. Under“Social Sciences”, the model reveals that “Anthropology” topics werelikely the topics of interest under this subcategory. Further, there waslikely interest in “civilization”, “culture”, and “Maya”. In thisregard, content inventory with other stories about ancientcivilizations, the Mayan Empire, or pyramids or temples could berecommended. Similarly, an advertising inventory with Machu Picchuadventure vacation trips, best airfare to Lima, Peru, or 5-star hotelsin Machu Picchu would get an increased conversion with users whoexhibited the interest profile shown in the topic model. Scoresindicating consumer disposition for an individual topic model categoriescan also be generated. A module can be used for generating scoresassociated with hierarchical categories. The scores may be used asindicators of consumer disposition as discussed above.

Turning to examples in market strategy, interested parties (e.g.,digital marketers and analyst) want to improve consumer analytics,prediction, and targeting accuracy. In this example, a URL from log dataof web content may be used to determine consumer disposition. The URLprovides context information that facilitates determining consumerdisposition. Context information can include an IP address of theconsumer who visited a site, a time the site was visited, and thespecific page visited. With reference to FIG. 5, the page URL 500 can befurther submitted to the classification engine component, in accordancewith embodiments of the present invention. It is contemplated that queryinformation may simply be indicated using an identifier (e.g., documentidentifier) of query content. As such, the entire topic modelhierarchical classification structure can be extracted and then used tofacilitate understanding of topics of interest. The output, as shown inFIG. 6, can indicate that clothing is a major topic for the webpage withsubcategories about jackets, coats, and suits and additional categoriesrelating to 1950s fashion and clothing brands. Additionally, becauseother articles in the knowledge corpus discuss related concepts, suchas, clothing jackets popular in the military and military jackets, thesetopic categories are also provided through analysis via the topic model.In this case, the analysis indicates a potential interest of theconsumer in jacket with a military cut or styling.

Similar to examples in digital publishing, the pages visited by aspecific user or user segment can be analyzed and modeled in a topicmodel having hierarchical classification structures. The topic model maybe for pages seen by a user for a particular date range. In this regard,moment-to-moment of aggregate consumer interest or purchase intent canbe determined based on analysis of associated web content. For example,a near-real time analysis based on a topic models of topic product orseries of topic product pages a user just visited would provide data tohelp efficiently predict products the user might be interested inbuying.

With reference now to FIG. 7, a flow diagram is provided thatillustrates a method 700 for performing analytics based on hierarchicalcategories in accordance with an embodiment of the present invention.Initially, at block 700, a plurality of features associated with adocument in the plurality of documents is identified. At block 712 adocument context is identified for analyzing the plurality of features.At block, 714, a hierarchical category path is generated from a selectedfeature from the plurality of features. Generating the hierarchicalcategory path may be based on analyzing the selected feature; theselected feature may be configured as feature vector in a feature vectorspace of the plurality of features, and identifying a shortest path fromthe feature vector to a root category associated with the documentcontext. In embodiments where multiple feature vectors are selected, a kshortest paths is identified from the selected feature vectors to rootcategories. At block 716, an index entry for the document associatedwith the selected feature and the hierarchical category path.

With reference now to FIG. 8, a flow diagram is provided thatillustrates a method 800 for performing analytics based on hierarchicalcategories in accordance with an embodiment of the present invention.Initially at block 802, a classification engine query is received. Theclassification engine query may include content or reference content tobe executed against a search engine with access to a categorizer index.At block 804, a plurality of features, associated with the contentreferenced by the classification engine query, are identified. Theplurality of features can be represented as feature vectors in a vectorspace of the plurality of features.

At block 806, a feature may be transmitted as a search engine query.When the feature is represented as a feature vector, a search enginequery feature vector is transmitted to the search engine. At block 816,a document is received; the document is identified based on the featureas the search engine query. It is contemplated that several documentsmay be identified, specifically, the documents may be identified usingthe search engine query feature vectors, and the documents areidentified based on a similarity between the search engine query featurevectors and feature vectors in the categorizer index.

At block 818, hierarchical category paths associated with the documentare identified. In embodiments, a topic model comprising thehierarchical category paths can be created. Upon creating the topicmodel, topic model scores associated with the hierarchical categorypaths are generated. The topic model comprising the hierarchicalcategory path and the path score is communicated for performing webanalytics. For example, an analytics application can be configured forreceiving the topic model of the classification query, analyzingenumerated category topics in the topic model, and automaticallygenerating additional web content for the web content associated withthe classification query, the additional web content is associated oneor more web marketing techniques such as segmenting, advertising, andtargeting.

Having briefly described an overview of embodiments of the presentinvention, an exemplary operating environment in which embodiments ofthe present invention may be implemented is described below in order toprovide a general context for various aspects of the present invention.Referring initially to FIG. 9 in particular, an exemplary operatingenvironment for implementing embodiments of the present invention isshown and designated generally as computing device 900. Computing device900 is but one example of a suitable computing environment and is notintended to suggest any limitation as to the scope of use orfunctionality of the invention. Neither should the computing device 900be interpreted as having any dependency or requirement relating to anyone or combination of components illustrated.

The invention may be described in the general context of computer codeor machine-useable instructions, including computer-executableinstructions such as program modules, being executed by a computer orother machine, such as a personal data assistant or other handhelddevice. Generally, program modules including routines, programs,objects, components, data structures, etc. refer to code that performparticular tasks or implement particular abstract data types. Theinvention may be practiced in a variety of system configurations,including hand-held devices, consumer electronics, general-purposecomputers, more specialty computing devices, etc. The invention may alsobe practiced in distributed computing environments where tasks areperformed by remote-processing devices that are linked through acommunications network.

With reference to FIG. 9, computing device 900 includes a bus 910 thatdirectly or indirectly couples the following devices: memory 912, one ormore processors 914, one or more presentation components 916,input/output ports 918, input/output components 920, and an illustrativepower supply 922. Bus 910 represents what may be one or more busses(such as an address bus, data bus, or combination thereof). Although thevarious blocks of FIG. 9 are shown with lines for the sake of clarity,in reality, delineating various components is not so clear, andmetaphorically, the lines would more accurately be grey and fuzzy. Forexample, one may consider a presentation component such as a displaydevice to be an I/O component. Also, processors have memory. Werecognize that such is the nature of the art, and reiterate that thediagram of FIG. 9 is merely illustrative of an exemplary computingdevice that can be used in connection with one or more embodiments ofthe present invention. Distinction is not made between such categoriesas “workstation,” “server,” “laptop,” “hand-held device,” etc., as allare contemplated within the scope of FIG. 9 and reference to “computingdevice.”

Computing device 900 typically includes a variety of computer-readablemedia. Computer-readable media can be any available media that can beaccessed by computing device 500 and includes both volatile andnonvolatile media, removable and non-removable media. By way of example,and not limitation, computer-readable media may comprise computerstorage media and communication media.

Computer storage media include volatile and nonvolatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer-readable instructions, data structures,program modules or other data. Computer storage media includes, but isnot limited to, RAM, ROM, EEPROM, flash memory or other memorytechnology, CD-ROM, digital versatile disks (DVD) or other optical diskstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other medium which can be used tostore the desired information and which can be accessed by computingdevice 900. Computer storage media excludes signals per se.

Communication media typically embodies computer-readable instructions,data structures, program modules or other data in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media includes wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, infrared and other wireless media. Combinations of any ofthe above should also be included within the scope of computer-readablemedia.

Memory 912 includes computer storage media in the form of volatileand/or nonvolatile memory. The memory may be removable, non-removable,or a combination thereof. Exemplary hardware devices include solid-statememory, hard drives, optical-disc drives, etc. Computing device 900includes one or more processors that read data from various entitiessuch as memory 912 or I/O components 920. Presentation component(s) 916present data indications to a user or other device. Exemplarypresentation components include a display device, speaker, printingcomponent, vibrating component, etc. I/O ports 918 allow computingdevice 900 to be logically coupled to other devices including I/Ocomponents 120, some of which may be built in. Illustrative I/Ocomponents 920 include a microphone, joystick, game pad, scanner,hard/soft button, touch screen display, etc.

Embodiments of the present invention have been described in relation toparticular embodiments which are intended in all respects to beillustrative rather than restrictive. Alternative embodiments willbecome apparent to those of ordinary skill in the art to which thepresent invention pertains without departing from its scope.

From the foregoing, it will be seen that this invention in one welladapted to attain all the ends and objects hereinabove set forthtogether with other advantages which are obvious and which are inherentto the structure.

It will be understood that certain features and sub-combinations are ofutility and may be employed without reference to other features orsub-combinations. This is contemplated by and is within the scope of theclaims.

What is claimed is:
 1. One or more computer storage media storingcomputer-useable instructions that, when executed by one or moreprocessors, causes the one or more processors to perform a method forperforming analytics based on hierarchical categories, the methodcomprising: receiving a classification engine query that references webcontent; identifying a first set of features associated with the webcontent referenced by the classification engine query; transmitting thefirst set of features as a search engine query; receiving documentsidentified based on the first set of features associated with the searchengine query, wherein the documents are identified based on a similaritybetween the first set of features of the search engine query and asecond set of features in a categorizer index, the categorizer indexcomprising (1) a plurality of categorized documents associated with thesecond set of features in the categorizer index and (2) hierarchicalcategory paths corresponding to the plurality of categorized documents;identifying a subset of the hierarchical category paths associated withthe documents, the subset of the hierarchical category paths comprisinga first hierarchical category path associated with a first search resultdocument and a second hierarchical category path associated with asecond search result document, wherein the subset of the hierarchicalcategory paths are identified based on a significance of the firsthierarchical category path and the second hierarchical category path tothe web content; and in response to identifying the first hierarchicalcategory path associated with the first search result document and thesecond hierarchical category path associated with the second searchresult document, creating a topic model comprising the subset of thehierarchical category paths associated with the documents identified inthe categorizer index based on a plurality of features associated withthe classification engine query and topic model scores associated withthe hierarchical category paths corresponding with the documentsidentified in the categorizer index wherein the topic model scores areindicative of consumer disposition.
 2. The media of claim 1, the methodfurther comprising: generating the topic model scores associated withthe subset of the hierarchical category paths of the topic model; andcommunicating the topic model comprising the subset of the hierarchicalcategory paths and corresponding path scores.
 3. The media of claim 2,wherein communicating the topic model further comprises: causing displayof the topic model as additional web content associated with the webcontent, wherein the additional web content facilitates navigation ofthe web content.
 4. The media of claim 2, wherein generating the topicmodel scores further comprises: computing a score for enumeratedcategory topics in the hierarchical category paths; and computing anindividual hierarchical category path score by summing the score of theenumerated categories in the individual hierarchical category path. 5.The media of claim 4, wherein the enumerated category topics in thehierarchical category path comprise a root category and a listing ofhierarchical subcategories that include a leaf category, wherein theroot category is associated with the individual hierarchical categorypath score and the hierarchical subcategories are associated with thescores for the enumerated category topics and a path number.
 6. Themedia of claim 1, the method further comprising: tagging the web contentwith hierarchical category path tags using tagging rules.
 7. The mediaof claim 6, the method further comprising: selecting at least a subsetof the hierarchical categories based at least in part on the scores ofenumerated category topics; identifying web content to tag withhierarchical categories from the subset of the hierarchical categories;and embedding the hierarchical categories from the subset ofhierarchical categories into metadata associated with the web content.8. The media of claim 7, wherein the tagging rules are based on one ormore of: weighting of selected enumerated category topics; thresholdscores for selecting enumerated category topics; and hierarchicalcategory path lengths.
 9. A computer-implemented method suitable for usein a computing environment utilizing a processor and a memory forperforming analytics based on hierarchical categories, the methodcomprising: receiving a classification engine query that references webcontent; identifying a first set of features associated with the webcontent referenced by the classification engine query; transmitting thefirst set of features as a search engine query; receiving documentsidentified based on the first set of features associated with the searchengine query, wherein the documents are identified based on a similaritybetween the first set of features of the search engine query and asecond set of features in a categorizer index, the categorizer indexcomprising a plurality of categorized documents associated with thesecond set of features in the categorizer index and hierarchicalcategory paths corresponding to the plurality of categorized documents;identifying a subset of the hierarchical category paths associated withthe documents, the subset of the hierarchical category paths comprisinga first hierarchical category path associated with a first search resultdocument and a second hierarchical category path associated with asecond search result document, wherein the subset of the hierarchicalcategory paths are identified based on a significance of the firsthierarchical category path and the second hierarchical category path tothe web content; and in response to identifying the first hierarchicalcategory path associated with the first search result document and thesecond hierarchical category path associated with the second searchresult document, creating a topic model comprising the subset of thehierarchical category paths associated with the documents identified inthe categorizer index based on a plurality of features associated withthe classification engine query and topic model scores associated withthe hierarchical category paths corresponding with the documentsidentified in the categorizer index wherein the topic model scores areindicative of consumer disposition.
 10. The method of claim 9, furthercomprising: generating the topic model scores associated with the subsetof the hierarchical category paths of the topic model; and communicatingthe topic model comprising the subset of the hierarchical category pathsand corresponding path scores.
 11. The method of claim 10, whereincommunicating the topic model further comprises: causing display of thetopic model as additional web content associated with the web content,wherein the additional web content facilitates navigation of the webcontent.
 12. The method of claim 10, wherein generating the topic modelscores further comprises: computing a score for enumerated categorytopics in the hierarchical category paths; and computing an individualhierarchical category path score by summing the score of the enumeratedcategories in the individual hierarchical category path.
 13. The methodof claim 12, wherein the enumerated category topics in the hierarchicalcategory path comprise a root category and a listing of hierarchicalsubcategories that include a leaf category, wherein the root category isassociated with the individual hierarchical category path score and thehierarchical subcategories are associated with the scores for theenumerated category topics and a path number.
 14. The method of claim 9,further comprising: tagging the web content with hierarchical categorypath tags using tagging rules.
 15. The method of claim 14, wherein thetagging comprises: selecting at least a subset of the hierarchicalcategories based at least in part on the scores of enumerated categorytopics; identifying web content to tag with hierarchical categories fromthe subset of the hierarchical categories; and embedding thehierarchical categories from the subset of hierarchical categories intometadata associated with the web content.
 16. The method of claim 14,wherein the tagging rules are based on one or more of: weighting ofselected enumerated category topics; threshold scores for selectingenumerated category topics; and hierarchical category path lengths. 17.A system for performing analytics based on hierarchical categories, thesystem comprising: a classification engine component configured for:receiving a classification engine query that references web content;identifying a first set of features associated with the web contentreferenced by the classification engine query; transmitting the firstset of features as a search engine query; receiving documents identifiedbased on the first set of features associated with the search enginequery, wherein the documents are identified based on a similaritybetween the first set of features of the search engine query and asecond set of features in a categorizer index, the categorizer indexcomprising (1) a plurality of categorized documents associated with thesecond set of features in the categorizer index and (2) hierarchicalcategory paths corresponding to the plurality of categorized documents;identifying a subset of the hierarchical category paths associated withthe documents, the subset of the hierarchical category paths comprisinga first hierarchical category path associated with a first search resultdocument and a second hierarchical category path associated with asecond search result document, wherein the subset of the hierarchicalcategory paths are identified based on a significance of the firsthierarchical category path and the second hierarchical category path tothe web content; and in response to identifying the first hierarchicalcategory path associated with the first search result document and thesecond hierarchical category path associated with the second searchresult document, creating a topic model comprising the subset of thehierarchical category paths associated with the documents identified inthe categorizer index based on a plurality of features associated withthe classification engine query and topic model scores associated withthe hierarchical category paths corresponding with the documentsidentified in the categorizer index wherein the topic model scores areindicative of consumer disposition.
 18. The system of claim 17, whereinthe classification engine component is further configured for:generating the topic model scores associated with the subset of thehierarchical category paths of the topic model; and communicating thetopic model comprising the subset of the hierarchical category paths andcorresponding path scores.
 19. The system of claim 17, furthercomprising: a categorizer index build component configured to perform:identifying a document context for the web content for analyzing theplurality of features; generating, for a selected feature of the firstset of features, a hierarchical category path using the document contextand the first set of features, wherein the hierarchical category path isan enumeration of a root category and corresponding one or moresubcategories, wherein the hierarchical category path is generated usinga category ontology of a knowledge corpus; and creating, in thecategorizer index, an index entry for the web content associated withthe selected feature and the hierarchical category path, wherein thecategorizer index comprises index entries that correspond to acategorized plurality of documents from the plurality of documents inthe knowledge corpus; and wherein the generated hierarchical path isidentified in the subset of the hierarchical category paths.
 20. Thesystem of claim 17, wherein the tagging comprises: selecting at least asubset of the hierarchical categories based at least in part on thescores of enumerated category topics; identifying web content to tagwith hierarchical categories from the subset of the hierarchicalcategories; and embedding the hierarchical categories from the subset ofhierarchical categories into metadata associated with the web content.